Resource Allocation and Multiagent Policy Formulation for Resource-Limited Agents Under Uncertainty

The problem of optimal policy formulation for teams of resourcelimited agents in stochastic environments is composed of two strongly coupled subproblems: a resource allocation problem and a policy optimization problem, both of which have individually received significant amount of attention. We show how to combine the two problems into a single constrained optimization problem that yields optimal resource allocations and policies that are optimal under these allocations. We model the stochastic environment as a multiagent Markov decision process, with social welfare of the group as the optimization criterion. We augment the standard MDP framework with constraints that ensure that the shared resource limitations are satisfied and formulate a constrained stochastic policy optimization problem that yields optimal policies among the class of realizable ones given the resource limitations. This work focuses on discrete operationalization resources that determine the actuating capabilities of the agents by defining the sets of actions available to them. We show that the problem of finding optimal policies under such constraints is NP-hard and present a solution algorithm based on mixed integer programming to solve the corresponding optimization problems.